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ABSTRACT 

This document reports the test of an experimental 
procedure designed by the Albany Medical College (AMC) to 
systematically reduce a large pool of applicants to one of manageable 
proportions for interviewing purposes. Data on nine predictor 
variables were coded and keypunched on 80-column cards for each 
applicant to September 1972 and 1973 admission. All applicants who 
obtained a predicted preclinical average of 85.4 were classified as 
••inter viewable;" those with an average below 80.0 were classified as 
••rejected" and applicants with averages between 80. o and 85.4 were 
"marginal." To test the efficacy of using the proposed analytic 
procedure for determining the interview status of applicants, a 
comparison was then made between screening results derived by 
computer and those of the AMC admissions screeners. Preliminary 
results indicated that nearly 40% savings in time has accrued from 
using this new procedure. AMC admissions screeners have found it 
easier to screen applicants which have been placed into selected 
categories beforehand. A 12-item bibliography is included. 
(Author/MJM) 



r 
h 

J- 

r^^- THE COMPUTER SCREENING OF MEDICAL SCHOOL APPLICANTS* 

ol 

f Robert J. Ambrosino and Paul L. Brading 



The Albany Medical College 



i Paper presented af the annual meetings of the American Educational 

Research Association, New Orleans, Louisiana - March, 1973 



*rhe authors would like to acknowledge the helpful contributions of 
Mr. Donald VanMeter • Director of Student Admissions, and Eugene 
Horn, Ph.D. • Dean of Admissions, without whose cooparation this 
study would not have been possible. 



us OEPARTMENTOF HEALTH 
EDUCATION & V/ELFARE 
OFFiCE OF EDUCATION 

THIS DOCUMENT HAS BEEN REP«0 
DUCED EXAC^iY AS RECEIVED FROM 
THE PERSON OR ORGANIZATION ORiG 
INATING IT POINTS OF VIEW OR OPIN 
IONS STATED DO NOT NECESSARILY 
REPRESENT O*"" aL OFFtCE OF EDU 
CATION POSITION OR POLICY 



A problem of major proportions facing the admissions departments of medical 
schools is the rapid growth in the number of first year applications as compared 
to the number of available first year places for study (Dube et« al.y 1971; Green, 
1972; Stritter et. al.j 1970, 1971). The Albany Medical College (AMC) has experien- 
ced over a 65% increase in the number of applications to first year study since the 
1969--70 applicant season. First year applications numbered over 3,000 for September 
1972 admission and over 4,000 for Septeniber 1973 admission. The completion of new 
and expanded medical school facilities in September of 1972 increased the number 
of places offered for first year study from 80 to 110. This overwhelmingly large 
ratio of number of applicants to number of mailable places for study surely de- 
mands a responsive and efficient system for processing applicant credentials. An 
important outcome of this system should be that highly desirable candidates are in- 
terviewed as soon as possible and clearly undesirable candidates are eliminated 
from consideration immediately. Thusly, not only would the likelihood of securing 
top quality students be enhanced, but also, a basis would be provided for examining 
in closer detail, those students with marginal records who might potentially re- 
present desirable candidates for stt^dy. An exploratory, collaborative study was un- 
dertaken with the AMC office of Admissions and Student Records to meet this chall- 
enge. Its purposes were: (a) to investigate the results of employing an analytic, 
computer-based methodology for examining the uadergraduate records of medical 
school applicants, with the end of determining interview status; and (b) to compare 
the computer derived results with the qualitative judgments of the AMC admissions 
screeners . 

Stepwise multiple regression procedures (Draper and Smith, 1966) were em- 
ployed to predict applicant preclinical (first two years) averages, which were in 
turn used as measures of interviewability. Multivariate methods have been used in 
previous studies to predict academic success (see Brading, 1971 for a review) and 
to aid admissions connittees in the selection of medical students (Best et. al.. 



1967, 1971; Conger and Fltz, 1963). The latter studies emphasized the use of stat- 
istically weighted measures for supplementing the subjective judgments of admiss- 
ions screeners in selecting students for medical study. The intent of the present 
study, however, was not primarily one of ranking students for purposes of selection. 
Rather, the goa! here was to test an experimental procedure for systematically 
reducing a large pool of applicants to one of manageable oroportions for interview- 
ing purposes. In this manner, a sequential strategy for determining the interview 
status of applicants was proposed. Specifically, the regression procedures vould 
be applied to the records of all applicants, placing them into three categories: 

(a) highly qualified applicants who ought to be interviewed as soon as possible; 

(b) applicants with clearly unacceptable credentials who ought to be rejected im- 
mediately; and (c) applicants with questionable records who should be considered 
in closer detail before a determination of interview status could be made. Records 
of applicants in categories (a) and (b) above usually manifest themselves quite 
clearly. Thus it was felt that applicants placed in these categories by the compu- 
ter procedures could be interviewed directly (or rejected immediately) without 
being subjected to the usual time consuming process of multiple screenings. (Prior 
to thif. study each applicant was humanly screened at least on two occasions, and 
in some cases, on three occasions, before a decision was rendered regarding his/ 
her interview status.) This projected savings in numbers of screenings would allow 
more time for the admissions screeners to direct their attention to students with 
questionable credentials. Consequently, this strategy would help to insure that 
highly qualified and clearly desirable candidates were afforded top priority in 
passing through the necessary admissions channels and that valuable screening time 
was not lost in examining th. records of clearly undesirable candidates. 



IT IS IMPORTANT TO NOTE THAT IN THIS STUDY NO DECISIONS REGARDING THE INTER- 
VIEW STATUS CF APPLICANTS WERE ACTUALLY MADE SOLELY ON THE BASIS OF THE COMPUTER 
DERIVED RESULTS. The reader will recall that one aim of the present study was 'o 
compare the computer derived screening results with those of the AMC admissions 
screeners. To this end each applicant was humanly screened at least once. 
Selection of Predictor Variables 

Initial selection of predictor variables was based on the findings of prev- 
ious prediction studies cited above, results of a questionnaire directed to medical 
schools regarding the use of predictive data for medical school admissions (Hamberg 
et. al. , 1971) and suggestions offered by the AMC Director of Student Admissions. 
An Initial pool of 15 predictor variables was entered Into a regression equation 
using preclinical (first two years) average as the criterion variable. The hypothe- 
sis or estimation-generating sample was comprised of students In the AMC graduating 
classes of 1969, 1970 and 1971; total sample size was 176. Five of the original 
variables, namely PREMEDICAL INSTRUCTORS RATINGS, SEX, MOTHERS OCCUPATION, MEDICAL 
COLLEGE ADMISSIONS TEST (MCAT) : GENERAL INFORMATION SUBTEST and MCAT: OVERALL were 

subsequently eliminated since these variables contributed only negligible incre- 

2 

ments to the multiple correlation coefficient, written R. One additional variable, 
UNDERGRADUATE SCIENCE GRADE POINT AVERAGE, was also eliminated from study since the 
relevant variance for this varinble appeared to be accounted for by two related 
variables, (OVERALL) UNDERGRADUATE GRADE POINT AVERAGE and REQUIRED COURSES 
AVERAGE. Table 1 contains a brief description of the remaining nine predictor var- 
iables which were included for study. R^ for these nine variables using the hypo- 
thesis-generating sample noted above was 0.54. Table 2 contains the raw regression 
coefficients which were derived for these data. Variable 6 (OVERALL) UNDEROtADUATE 
GRADE POINT AVERAGE received the largest estimation weight, a result which is con- 
sistent with the widely drawn conclusion that the single best predictor of future 
academic success is past academic performance. 



INSERT TABLES 1 AND 2 ABOUT HERE 



Procedure 

Data on the nine predictor variables were coded and keypunched on 80-column 
hollerith cards for each applicant to September 1972 and September 1973 admission. 
Estimation weights derived using the hypothesis-generating sample were applied to 
the coded scores of each applicant, resulting In a predicted preclinical average. 
The mean predictedprecllnlcal average (85.4) computed for the total pool of appll* 
cants *ho were accepted for September 1971 admission was chosen as the cut-!off 
poJ-it for determining those students who were to be classified as litterviewable. 
That is, all applicants who obtained a predicted average of 85.4 or higher were 
classified as "Intervlewable". Further, applicants whose predicted average fell 
below 80.0 were classified as "rejects". Applicants whose predicted average fell 
between 80.0 and 85.4 were classified as "marginal'*, Indicating that a closer, more 
detailed examination of their records was required before a determination of in- 
terview status could be made. Each applicant was also humanly screened at least 
once by one of the AMC admissions screeners. To test the efficacy of using the pro- 
posed analytic procedure for determining the interview status of applicants, a com- 
parison was then Liade between screening results derived by computer and those of the 
AMC admissions screeners • 
Results and Discussion 

Figure 1 depicts the percent agreement between screening results based on the 
statistically weighted measures and tho«e of the Admissions Committee screening 
procedures. Data summarized in the figure are based on results for September 1972 
admissions. Data for September 1973 admissions were not completely analyzed at the 
time of this writing, however, a partial comparison yielded a similar configura- 
tion. Reading from left to right, entriea 1^, the top and bottom halves of the 



figure sum to 100 per cent. This corresponds to a comparison of the results given 
by the two procedures using the analytic procedure as the control variable in each 
case. Entries in the upper right-hand and lower left-hand quadrants were designa- 
ted as "hits" since screening results from both procedures were in agreement. As 
can be seen, the extent of agreement in the hits category ranged from 63.67. to 
78.271, a preliminary result which was encouraging. Entries in the upper left-hand 
quadrant were labeled as "false positives" (Cronbach, 1960). This category in- 
cluded those cases in which the analytic procedure classified applicants as in- 
terviewable and the Admissions Committee screening procedures classified appli- 
cants as non-interviewable. Two hypotheses were advanced to explain this apparent 
screening discrepancy. In some cases applicants with near marginal records also 
had uncommonly high entering ages, this fact alone spuriously inflating the value 
of the predicted criterion to exceed the cut-off point. Consequently, the analytic 
method classified these applicants as interviewable, while the Admissions Committee 
screening procedures classified them as non-interviewable. In other cases it was 
felt that certain geographical and undergraduate college considerations relative 
to class balance influenced the decisions of the Admissions Committee screeners in 
classifying applicants as non-interviewable. 

INSERT FIGURE 1 ABOUT HERE 

Entries in the lower right-hand quadrant were labeled as "misses". This ca- 
tegory included cases in which the analytic procedure indicated a non-interview 
status and the Admissions Committee screening procedures indicated an interview 
status. Three hypotheses were advanced to explain this apparent screening dis- 
crepancy. In some cases students who had demonstrated generally poor academic 
performance in the lirst two years of undergraduate study, but greatly improved 
performance in the last two years of undergraduate study, vere considered by the 
Admissions Committee screeners as potentially good candidates and consequently, in- 
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Cerviewable. In other cases » students who had demonstrated marginal academic re- 
cords on the undergraduate level » but who had excelled on the graduate level » were 
classified as interviewable by Admissions Committee screening standards, and non- 
interviewable by the analytic procedure • In still other cases , applicants with 
special dispositions (i.e,, alumni children, expressed faculty interest, minority 
group status, etc,) were afforded additional careful sct^itlny by the Admiss- 
ions Committee screeners in the determination of interview status. This of course, 

I 

was not possible in the case of the analytic procedure. 

Another source of error which was believed to be potentially responsible for 
some of these discrepant cases was a low inter-rater reliability among the judg- 
ments of th^ Admissions Committee screeners. The degree to which low inter-rater 
reliability affected the results of this study, however, appeared to be inconse- 
quential. An analysis of inter-rater reliabilities among those cases requiring 
multiple screenings yielded Pearson-rs in the low to middle .80s. 

Findings such as those summarized above offered important suggestions regatu- 
ing the adjustments which are needed to improve the accuracy of the prediction pro- 
cedure. For example, last two years grade point average may be an important variable 
to include in the set of predictor variables. Also, certain prescribed indicators 
should be included in the computer program printout to flag students who have 
special dispositions, graduate school records and above average entering age. 

i 

Conclusions 

An experimental, analytic computer-based methodology for determining the in- 
terview status of medical school applicants was tested in a limited setting and 
provided encouraging results. As a consequence of comparing the results of two al- 
ternate strategies for determining the inter viewability of applicants, certain re- 
finements were indicated which would improve the predictive accuracy of the ex- 
perimental procedure. Preliminary results indicated that nearly a 40% savings in 
r^J^^ time has accrued from using this new procedure. AMC admissions screeners have found 



it easier to screeri applicants vhich have been placed into selected categories be- 
forehand* An apparent drawback in this process, however, is the amount of time re- 
quired to code and keypunch the data necessary for the computer program. Problems 
such as these might easily be resolved by using special forms designed to be dir- 
ectly converted into card images by electronic processing (see Rankin et. al, 1972). 

Although additional experimentation with the methods reported in this study 
is indicated before general applications can be recommended, results of this study 
have shown the potential usefulness of employing analytic methods as a valuable aid 
in processing the undergraduate student records of medical school applicants. 
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TABLE 1 



Number/Code 



DESCRIPTION OF PREDICTOR VARIABLES INCLUDED FOR STUDY 



Descriotion 



1. AGE 

2. COLRNK 



3. MCATVB 

4. MCATQN 

5. MCATSC 

6. UGA 

7. FATHOC 

8. EXTCUR 

9. RQDCRS 



Age of applicant at time of application. 

Internal mechanism for ranking undergraduate 
colleges, based on Medical College Admissions 
Test (MCAT) average scores earned by students 
dt that institution* 

Verbal subtest of the MCAT. 

Quantitative subtest of the HC^T. 

Science subtest of the MCAT. 

(Overall) undergraduate grade point average of 
applicant* 

Applicants fathers occupation » based on a 7-point 
scale* 

Undergraduate extra-curricular activities of 
applicant, based on a 12-point scale* 

Grade point average of selected courses required 
for admission, including introductory biology, 
chemistry and physics and organic chemistry* 



TABLE 2 



RAH REGRESSION COEFFICIENTS DERIVED FOR 
NINE PREDICTOR VARIABLES (R^ = 0.54) 



Tf— 

Variable Coefficient 



AGE 

COLRNK 

MCATVB 

MCATON 

MCATSC 

UGA 

FATHOC 

EXTCUR 

RODCRS 



0.2830 
0.1028 

-0.0068 
0.0060 
0.0066 
4.2160 
0.1363 

-0.2251 
0.4631 



Comouter procedures-yes 



AMC screeners 
no 



36. A% 


63.6% 


78.2% 


21.8% 



AMC screeners 
yes 



Computer procedures-no 



Figure 1 - Percentage agreement between screening results - based on a comparison 
of resuxts given by the analytic computer-based method and those of the 
AMC admissions screeners. 
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